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YbMgGa 04 , a structurally perfect two-dimensional triangular lattice with odd number of electrons 
per unit cell and spin-orbit entangled effective spin-1/2 local moments of Yb®"*" ions, is likely to 
experimentally realize the quantum spin liquid ground state. We report the first experimental 
characterization of single crystal YbMgGa 04 samples. Due to the spin-orbit entanglement, the 
interaction between the neighboring Yb^"*" moments depends on the bond orientations and is highly 
anisotropic in the spin space. We carry out the thermodynamic and the electron spin resonance 
measurements to confirm the anisotropic nature of the spin interaction as well as to quantitatively 
determine the couplings. Our result is a first step towards the theoretical understanding of the 
possible quantum spin liquid ground state in this system and sheds new lights on the search of 
quantum spin liquids in strong spin-orbit coupled insulators. 

PACS numbers: 75.10.Kt, 75.30.Et, 75.30.Gw, 76.30.-v 


Introduction .—Recent theoretical advance has ex¬ 
tended the Hastings-Oshikawa-Lieb-Schultz-Mattis theo¬ 
rem to the spin-orbit coupled insulators W- It is shown 
that as long as the time reversal symmetry is preserved, 
the ground state of a spin-orbit coupled insulator with 
odd number of electrons per unit cell must be exotic [1]. 
This important result indicates that the ground state of 
strong spin-orbit coupled insulators can be a quantum 
spin liquid (QSL). QSLs, as we use here, are new phases 
of matter that are characterized by properties such as 
quantum number fractionalization, intrinsic topological 
order, and gapless excitations without symmetry break¬ 
ing [SI H]. Among the existing QSL candidate materi¬ 
als [7H33], the majority have a relatively weak spin-orbit 
coupling (SOC), which only slightly modifies the usual 
SU(2) invariant Heisenberg interaction by introducing 
weak anisotropic spin interactions such as Dzyaloshinskii- 
Moriya interaction |MIt36j . It is likely that the QSL 
physics in many of these systems mainly originates from 
the Heisenberg part of the Hamiltonian rather than 
from the anisotropic interactions due to the weak SOC. 
The exceptions are the hyperkagome Na 4 lr 30 g and the 
pyrochlore quantum spin ice materials where the non- 
Heisenberg spin interaction due to the strong SOC plays 
a crucial role in determining the ground state proper¬ 
ties [M [m IMSg, though both systems contain even 
number of electrons per unit cell. Therefore, it is de¬ 
sirable to have a QSL candidate system in the spin-orbit 
coupled insulator that contains odd number electrons per 



FIG. 1. (Golor online.) The YbMgGa 04 lattice structure (a) 
and the triangular lattice in the ab plane (b) formed by the 
Yb®+ ions. The inset defines the coordinate system for the 
spin components. 


unit cell, where the strong SOC leads to a non-Heisenberg 
spin Hamiltonian jSTJ [351 SOI I35H53] . 

In this Letter, we propose a possible experimental re¬ 
alization of the QSL with strong SOC and odd number 
of electrons per unit cell in YbMgGa 04 , where the Yb®+ 
ions form a perfect triangular lattice (see Fig. [^. It was 
previously found in a powder sample that the system has 
a Curie-Weiss temperature — —4K but does not 

order magnetically down to 60mK [S3]. To understand 
the nature of the obviously disordered ground state ob¬ 
served in YbMgGa 04 , it is necessary to have a quantita¬ 
tive understanding of the local moments and microscopic 
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Hamiltoninan. We here confirm the effective spin-1/2 
nature of the Yb^“'" local moments at low temperatures 
from the heat capacity and the magnetic entropy mea¬ 
surements in high-quality single crystal samples. Because 
the ion contains odd number of electrons, the ef¬ 

fective spin is described by a Kramers’ doublet. Based 
on this fact, we theoretically derive the symmetry al¬ 
lowed spin Hamiltonian that is non-Heisenberg-like and 
involves four distinct spin interaction terms because of 
the strong SOC. Combining the spin susceptibility results 
along different crystallographic directions and the elec¬ 
tron spin resonance (ESR) measurements in single crystal 
samples, we quantitatively confirm the anisotropic form 
of the spin interaction. We argue that the QSL physics 
in YbMgGa 04 may originate from the anisotropic spin 
interaction. To our knowledge, YbMgGa 04 is probably 
the hrst strong spin-orbit coupled QSL candidate system 
that contains odd number of electrons per unit cell with 
effective spin-1/2 local moments. 

Experimental technique. —High-quality single crystals 
1cm) of YbMgGa 04 , as well as the non-magnetic iso- 
structural material LuMgGa 04 [51], are synthesized by 
the floating zone technique. X-ray diffractions (XRD) 
are performed on the cutting single crystals to confirm 
the crystallization, the crystallographic orientation and 
the absence of the impurity phase, and for the sin¬ 
gle crystal structure refinements [55]. The high qual¬ 
ity of the crystallization was confirmed by the narrow 
XRD rocking curves with A20 ~ 0.06° and 0.04° on 
ab planes for YbMgGa 04 and LuMgGa 04 crystals, re¬ 
spectively. Magnetization (~ 60mg of YbMgGa 04 single 
crystals) and heat capacity measurements (10 ~ 20mg 
of YbMgGa 04 and LuMgGa 04 single crystals) were per¬ 
formed using a Quantum design physical property mea¬ 
surement system along and perpendicular to the c axis 
at 1.8 ~ 400K under 0 ^ 14T. The magnetic susceptibil¬ 
ities of single crystals agree with that of powder samples, 
X||/3-|-2 x_l/ 3 ~ XPowder- The ESR measurements 
60mg of YbMgGa 04 single crystals) at 1.8 ^ 50K along 
different crystallographic orientations were performed us¬ 
ing a Bruker EMX plus 10/12 GW-spectrometer at X- 
band frequencies (f ~ 9.39GHz); the spectrometer was 
equipped with a continuous He gas-flow cryostat. 

Kramers’ doublet and exchange Hamiltonian. —The 
Yb3+ ion in YbMgGa 04 has an electron configuration 
4/^3^ ajift from the Hund’s rules the orbital angular mo¬ 
mentum [L = 3) and the spin (s = 1/2) are entangled, 
leading to a total angular momentum J = 7/2. Un¬ 
der the trigonal crystal electric held, the eight-fold de¬ 
generate J = 7/2 states are splitted into four Kramers’ 
doublets [55Mn H5] . By htting the heat capacity results 
with an activated behavior, we hnd the local ground state 
doublet is well separated from the hrst excited doublet by 
an energy gap A ~ 420K. This indicates that only the 
local ground state doublet is active at T ^ A. More¬ 
over, the magnetic entropy reaches to a plateau at Rln 2 



FIG. 2. (Color online.) (a, b) The magnetization of the 
YbMgGa 04 single crystals measured at 10, 6, 3 and 1.9K. 
The dashed lines are linear hts of the experimental results 
for fields above 12T at 1.9K. The solid curves are the cor¬ 
responding magnetization calculated by the molecular field 
approximation, (c, d) The inverse spin susceptibilites (af¬ 
ter subtracting the Van Vleck paramagnetism) htted by the 
Curie-Weiss law (in dashed lines) for the YbMgGa 04 single 
crystals. 


per mol Yb^"*- around 40K, which is consistent with the 
thermalization of the 2-fold degenerate ground state dou¬ 
blet [53] [51] . 

As it is analogous to the local moments in the py- 
rochlore ice systems EH, one can introduce an effective 
spin-1/2 degree of freedom, S^, that acts on the local 
ground state doublet. The low-temperature magnetic 
properties are fully captured by these effective spins. Be¬ 
cause the 4/ electron is very localized spatially [33], it is 
sufficient to keep only the nearest-neighbor interactions 
in the spin Hamiltonian [56]. Via a standard symme¬ 
try analysis, we find the generic spin Hamiltonian that 
is invariant under the R3m space group symmetry of 
YbMgGa 04 is given by 

n = + J±{sts- + S-S+) 

(V) 

+ (* o j))], (1) 

where Sf = Sf ± iSf , and the phase factor = 
1 , gi2ir/3, g-i27r/3 bond ij along the ai,a 2 ,a 3 di¬ 

rection (see Fig. [^, respectively. This generic Hamil¬ 
tonian includes all possible microscopic processes that 
contribute to the nearest-neighbor spin interaction. The 
highly anisotropic spin interaction in "H is a direct 
consequence of the spin-orbit entanglement in the lo¬ 
cal ground state doublet. Moreover, the antisymmet¬ 
ric Dzyaloshinskii-Moriya interaction is prohibited in the 
Hamiltonian because of the inversion symmetry. 

Magnetization and magnetic susceptibility. —In order 
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to quantitatively determine the exchange couplings, we 
first perform the magnetization measurements for the 
YbMgGa 04 single crystals down to 1.8K under an exter¬ 
nal magnetic field (from OT to 14T) parallel and perpen¬ 
dicular to the c axis (see Fig. [^. The Zeeman coupling 
to the external field is also constrained by the lattice 
symmetry and is given by |57j 

Hz = -MOMS E + 9\\hi\S^] ■ (2) 

i 

As shown in Fig. the magnetization processes are ob¬ 
tained for both field directions at 1.9K. When the field is 
above 12T, both magnetizations saturate and become lin¬ 
early dependent on the field. The slope of the M-H curve 
is temperature-independent and is understood as the 
Van Vleck susceptibility (x^ “ 0.118(2)cm^/mol-Yb^''", 
xY^ = 0.0479(8)cm^/mol-Yb^''') that arises from the 
field-induced electronic transitions [SS]. After subtract¬ 
ing the Van Vleck paramagnetic contribution, we obtain 
the saturated magnetic moments ((/||/iB/2 and (/_l^b/ 2), 
from which we extract the g factors gy = 3.721(6), 5 _l = 
3.060(4) [M]. 

We apply a small external field (O.OIT) to measure 
the spin susceptibilities parallel and perpendicular to the 
c axis as a function of temperature. At high tempera¬ 
tures (T Y 8K) both susceptibilities (after the substrac- 
tion of Van Vleck paramagnetism) are well fitted by the 
Curie-Weiss law (see Fig. [^. From the spin Hamilto¬ 
nian, it is ready to obtain the Curie-Weiss temperatures 
®cw = ~3 Jz^/ 2 (0CW = ~3J±) for the field parallel 
(perpendicular) to the c axis. We then use the above 
relations to find Jzz and J±. Alternatively, we apply 
the high-temperature molecular field approximation to 
fit the field dependence of magnetizations. As shown in 
Fig-i the molecular field result agrees with the experi¬ 
ments very well at lOK. These two approaches together 
yield Jzz = 0.98(8)K and J± = 0.90(8)K. 

ESR. —The remaining two coupling constants, J±± 
and Jz±, that contribute to the anisotropic spin inter¬ 
action, completely break the U(l) spin rotation but keep 
time reversal symmetry intact. They cannot be well re¬ 
solved by the above thermodynamic measurements. To 
precisely determine them, we apply the exhaustive ESR 
measurements and analyze the ESR linewidths. It is 
well-known that the ESR linewidth is a powerful and 
direct measure of the anisotropic spin interactions [H- 
[M] . We perform the ESR measurements from 1.8K to 
50K along different crystallographic orientations, where 
the wide ESR signals, as broad as fioAH{6) ~ 0.4T, were 
observed (see Fig. 00 and raw ESR signals [Ml)- 

Here we discuss various sources that broaden the 
ESR linewidth. The first one is the hyperfine in¬ 
teractions that contribute to the ESR linewidth with 
HoAHh ~ |A||P/( 5 ^b|J ol) 2mT |S3], where the hy¬ 
perfine coupling, |A|||, is about 2CHz for Yb^+ [55] . 



FIG. 3. (Color online.) The temperature dependence of ESR 
linewidths (a) parallel and (b) perpendicular to the c axis. 
The dashed lines are the corresponding constant fits to the 
ESR linewidth data at T > 6K. (c) The deviation, Rp, of the 
experimental ESR linewidthes from the theoretical ones for 
YbMgGa 04 . The dashed rectangle gives the optimal param¬ 
eters I J±±| = 0.155(9)K and \ Jz±\ ~ 0.04(10)K. 


and Jo is the isotropic Heisenberg coupling defined as 
Jo = (4J± -b Jz^)/3 - 1.5(1)K in Eq. The 

next-nearest-neighbor magnetic dipole-dipole interac¬ 
tions also broaden the ESR signal with fioAHd ~ 
\Ed\'^/igg-B\Jo\) 0.3mT [ 53 ]. Here, we have made a 
maximal estimate of the next-nearest-neighbor dipole- 
dipole interaction \Ed\ as fdog^ where a is 
the lattice constant. All the Yb^"*" ions share the same g- 
tensor, the Zeeman interaction is homogeneous and thus 
does not contribute to the ESR linewidth [ 53 ] . All the 
above contributions together give an ESR linewidth that 
is two orders of magnitude smaller than the observed 
value. To account for such a large ESR linewidth that is 
~ 0.4T, the only remaining origin lies in the anisotropy 
of the nearest-neighbor spin interaction. 

We now decompose the spin Hamiltonian in Eq. Q 
into the isostropic and the anisotropic parts 

H = JoY.S,-Sj+H' (3) 

(bi) 


where Jq was previously introduced, Fy is a trace¬ 
less coupling matrix, and H' = J2{ij) 'Ffis the 
anisotropic part of the spin interaction. With the Zee- 
man term in Eq. ([^, the ESR linewidth is obtained as 


AH{9) 


g.Bg{0) ^ M4 




where 6 is the angle between the external field and 
the c axis, g{6) = {g'^^cos^ 9 + g^sir? 6)^/'^, M 2 = 
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,%'])/{M'^M ) is the second moment, 
and Mi = {[%,[%', M+]][n,[n',M-]])l{M+M-) is the 
fourth moment [62]. Here, 5'^^. 

The ESR signal of YbMgGa 04 single crystal can be 
well fitted by the first-derivative Lorentzian line shape 
with a small contribution of dispersion as described by 
Ref. EH Both /xoAi7||(T) and ^oAH±{T) show a gradual 
broadening [61j with increasing temperature for ksT < 
5 Jq, and reach almost temperature-independent maxima 
at /ioAiJii = 0.4352(9)T and fioAHj_ = 0.4152(7)T for 
ksT > 5Jo (see Fig. [^. We fit these high-temperature 
ESR linewidths according to the theoretical results. In 
Fig-i we plot the deviation of the experimental result 
from the theoretical one. 


1 r 

Rp = - 


r A77|| - AHf^ 


AHj_ - Ai7“' 

[ Ai7|| 


AH± 


( 5 ) 


as a function of J±± and Jz±- The optimal fitting is 
obtained by setting |J±±| = 0.155(3)K and \Jz±\ = 
0.04(8)K whose signs cannot be fixed by the fitting. 

As an unbiased check of the fitted results, we use the 
optimal couplings to calculate the angle dependence of 
the ESR linewidth, where 0 is the angle 

between the external field and the c axis. As shown 
in Fig. 1^ the experimental curve agrees with the the¬ 
oretical result very well. Moreover, we apply the high- 
temperature series expansions to compute the spin sus¬ 
ceptibilities per Yb^+ ion up to 


XII 


X-L 


iJzz 3 J|. -I- Jj.j- -I- 
AkeT ^ ~ 2kBT ~ 2fc|r2 
15J2 ^ 

dfcsT ^ ksT klT^ klT^ 
5</-,^ Ji-Jz-z , 

“ 16fc|r2 “ 8fc|T2 “ 


( 6 ) 


( 7 ) 


As we depict in Fig. [^ the high-temperature expansion 
shows an better fitting with the experimental results at 
lower temperatures than the simple Curie-Weiss laws. 

Discussion .—In the previous powder sample measure¬ 
ments, the magnetic heat capacity of YbMgGa 04 be¬ 
haves as cx (7 « 2/3) from about IK down to 
0.06K (SSj, suggesting the system is probably in a gap¬ 
less QSL phase [53]. The residual magnetic entropy of the 
system at 0.06K is less than 0.6% of the total magnetic 
entropy |53] . This is a strong indication that we are in¬ 
deed accessing the ground state property. As far as we are 
aware of, this is the first clear observation of Cy oc 
in QSL candidate systems. In fact, this behavior is com¬ 
patible with what one may expect for the U(l) QSL with 
a spinon Fermi surface in two dimensions |67H69j , a state 
previously proposed for the organic k-(ET) 2 Cu 2 (CN )3 
and EtMe 3 Sb[Pd(dmit) 2]2 pTHMj . Although alternative 



FIG. 4. (Color online.) (a) Angular dependence of ESR 
linewidth. The dashed curve is the calculated ESR linewidth. 
(b) The magnetic susceptibilities of YbMgGa 04 single crys¬ 
tal after subtracting the Van Vleck paramagnetism. The 
solid curves are the calculated susceptibilities using the high- 
temperature series expansion. The dashed curves are the 
Curie-Weiss susceptibilities. 


proposals also exist m, the QSL physics in the organics 
is believed to originate from the strong charge fluctua¬ 
tion of the weak Mott regime that induces a sizable ring 
exchange and thus destabilizes the 120 ° magnetic order 
for a triangular system [67H69j . In contrast, the physical 
mechanism to realize possible QSL in YbMgGa 04 should 
be rather different. The / electrons of YbMgGa 04 are 
very localized and are in the strong Mott regime. The 
charge fluctuation is very weak and the ring exchange 
process should be negligible. On the other hand, the 
anisotropic J±± and Jz± spin interaction is a new ingre¬ 
dient brought by the spin-orbit entanglement of the Yb 
/ electrons and is expected to be the physical origin of 
the QSL physics. This is because in the absence of the 
anisotropic J±± and Jz± spin interaction the antiferro¬ 
magnetic XXZ model would produce a conventional mag¬ 
netic order m- It is the anisotropic J±± and Jz± spin 
interaction that competes with the XXZ model and may 
melt the magnetic order in certain parameter regime m- 
Through the current single crystal measurements, we ex¬ 
pect YbMgGa 04 to be a spin-orbit coupled QSL in which 
the anisotropic spin interaction is the driving force. 

To summarize, we have characterized the magnetic 
properties of large YbMgGa 04 single crystals that are 
grown for the first time. The crystal structure and effec¬ 
tive spin-1/2 Hamiltonian of YbMgGa 04 are precisely 
determined by single crystal X-ray diffractions, thermo¬ 
dynamic measurements and ESR linewidths on the ori¬ 
entated single crystals. We find that the anisotropic spin 
exchange interaction on the Yb triangular lattice signifi¬ 
cantly broadens the ESR linewidths. We argue that the 
anisotropic spin interaction plays an important role to 
stabilize the possible QSL ground state in YbMgGa 04 . 
In the future, it will be interesting to numerically study 
the theoretical model in our work, classify QSLs in strong 
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spin-orbit coupled insulators, and use inelastic neutron 
scattering to detect the possible fractionalized spin exci¬ 
tation in the single crystal samples. 
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